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Abstract 

A long and lasting problem in agent research has been to close the 
gap between agent logics and agent programming frameworks. The main 
reason for this problem of establishing a link between agent logics and 
agent programming frameworks is identified and explained by the fact 
that agent programming frameworks have not incorporated the concept of 
a declarative goal. Instead, such frameworks have focused mainly on plans 
or goals-to-do instead of the end goals to be realised which are also called 
goals-to-be. In this paper, a new programming language called GOAL is 
introduced which incorporates such declarative goals. The notion of a 
commitment strategy - one of the main theoretical insights due to agent 
logics, which explains the relation between beliefs and goals - is used to 
construct a computational semantics for GOAL. Finally, a proof theory 
for proving properties of GOAL agents is introduced. Thus, we offer 
a complete theory of agent programming in the sense that our theory 
provides both for a programming framework and a programming logic 
for such agents. An example program is proven correct by using this 
programming logic. 

1 Goal- Oriented Agent Programming 

Agent technology has come more and more into the limelight of computer 
science. Intelligent agents have not only become one of the central topics 
of artificial intelligence (nowadays sometimes even defined as "the study of 
agents", Q), but also mainstream computer science, especially software en- 
gineering, has taken up agent-oriented programming as a new and exciting 
paradigm to investigate, while industries experiment with the use of it on a 
large scale, witness the results reported in conferences like Autonomous Agents 
(e.g. |8)) and books like e.g. 

Although the definition of an agent is subject to controversy, many re- 
searchers view it as a software (or hardware) entity that displays some form 



of autonomy, in the sense that an agent is both reactive (responding to its en- 
vironment) and pro-active (taking initiative, independent of a user). Often this 
aspect of autonomy is translated to agents having a mental state comprising (at 
least) beliefs on the environment and goals that are to be achieved (p5[). 

In the early days of agent research, an attempt was made to make the concept 
of agents more precise by means of logical systems. This effort resulted in a 
number of - mainly - modal logics for the specification of agents which formally 
defined notions like belief, goal, intention, etc. associated with agents ||2^, |l^, ^ 
^. The relation of these logics with more practical approaches remains unclear, 
however, to this day. Several efforts to bridge this gap have been attempted. 
In particular, a number of agent programming languages have been developed 
to bridge the gap between theory and practice ||2j, [l4| . These languages show 
a clear family resemblance with one of the first agent programming languages 
Agent-0 1^, and also with the language ConGolog 0, 

These programming languages define agents in terms of their corresponding 
beliefs, goals, plans and capabilities. Although they define similar notions as 
in the logical approaches, there is one notable difference. In logical approaches, 
a goal is a declarative concept, (also called a goal-to-be), whereas in the cited 
programming languages goals are defined as sequences of actions or plans (or 
goals-to-do). The terminology used differs from case to case. However, whether 
they are called commitments (Agent-0), intentions (AgentSpeak |24 ), or goals 
(3APL [^) makes little difference: all these notions are structures built from 
actions and therefore similar in nature to plans. With respect to ConGolog, a 
more traditional computer science perspective is adopted, and the corresponding 
structures are simply called programs. The PL AC A language [Q, a successor of 
AGENTO, also focuses more on extending AGENTO to a language with complex 
planning structures (which are not part of the programming language itself!) 
than on providing a clear theory of declarative goals of agents as part of a 
programming language and in this respect is similar to AgentSpeak and 3APL. 
The type of goal included in these languages may also be called a goal-to-do and 
provides for a kind of procedural perspective on goals. 

In contrast, a declarative perspective on goals in agent languages is still miss- 
ing. Because of this mismatch it has not been possible so far to use modal logics 
which include both belief and goal modalities for the specification and verifica- 
tion of programs written in such agent languages and it has been impossible to 
close the gap between agent logics and programming frameworks so far. The 
value of adding declarative goals to agent programming lies both in the fact 
that it offers a new abstraction mechanism as well as that agent programs with 
declarative goals more closely approximate the intuitive concept of an intelli- 
gent agent. To fully realise the potential of the notion of an intelligent agent, 
a declarative notion of a goal, therefore, should also be incorporated into agent 
programming languages. 

In this paper, we introduce the agent programming language GOAL (for 
Goal-Oriented Agent Language), which takes the declarative concept of a goal 
seriously and which provides a concrete proposal to bridge the gap between 
theory and practice. GOAL is inspired in particular by the language UNITY 
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designed by Chandy and Misra Q , be it that GOAL incorporates complex agent 
notions. We offer a complete theory of agent programming in the sense that our 
theory provides both for a programming framework and a programming logic 
for such agents. In contrast with other attempts Q to bridge the gap, our 
programming language and programming logic are related by means of a formal 
semantics. Only by providing such a formal relation it is possible to make sure 
that statements proven in the logic concern properties of the agent. 

2 The Programming Language GOAL 

In this section, we introduce the programming language GOAL. As mentioned 
in the previous section, GOAL is influenced by by work in concurrent program- 
ming, in particular by the language UNITY (0). The basic idea is that a set 
of actions which execute in parallel constitutes a program. However, whereas 
UNITY is a language based on assignment to variables, the language GOAL is 
an agent-oriented programming language that incorporates more complex no- 
tions such as belief, goal, and agent capabilities which operate on high-level 
information instead of simple values. 

2.1 Mental States 

As in most agent programming languages, GOAL agents select actions on the 
basis of their current mental state. A mental state consists of the beliefs and 
goals of the agent. However, in contrast to most agent languages, GOAL in- 
corporates a declarative notion of a goal that is used by the agent to decide 
what to do. Both the beliefs and the goals are drawn from one and the same 
logical language, C, with associated consequence relation \=c- In this paper, 
yC is a propositional language, and one may think about |=c' as 'classical con- 
sequence'. In general however, the language C may also be conceived as an 
arbitrary constraint system, allowing one to combine tokens (predicates over a 
given universe) using the operator A (to accumulate pieces of information) and 
3^ (to hide information) to represent constraints over the universe of discourse 
(Cf. iQ). In such a setting, one often assumes the presence of a constraint 
solver that tests T \=c f, i.e., whether information F entails ip. 

Our GOAL-agcnt thus keeps two databases, respectively called the belief 
base and the goal base. The difference between these two databases originates 
from the different meaning assigned to sentences stored in the belief base and 
sentences stored in the goal base. To clarify the interaction between beliefs and 
goals, one of the more important problems that needs to be solved is establishing 
a meaningful relationship between beliefs and goals. This problem is solved 
here by imposing a constraint on mental states that is derived from the default 
commitment strategy that agents use. The notion of a commitment strategy 
is explained in more detail below. The constraint imposed on mental states 
requires that an agent does not believe that (p is the case if it has a goal to 
achieve (j), and, moreover, requires 4> to be consistent if (/> is a goal. 
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Definition 2.1 (mental state) 

A mental state of an agent is a pair (S,r) where E C £ are the agent's behefs 
and T C jC are the agent's goals (both sets may be infinite) and S and F are 
such that: 

• S is consistent (S false) 

• r is such that, for any 7 e F: 

(«) 7 is not entailed by the agent's beliefs (S 7), 
{it) 7 is consistent {Y=c ^l), and 

(iii) for any 7', if 7 ^ 7' and 7' satisfies (i) and (ii) above, then 

7' er 

A mental state does not contain a program or plan component in the 'classi- 
cal' sense. Although both the beliefs and the goals of an agent are drawn from 
the same logical language, as we will see below, the formal meaning of beliefs 
and goals is very different. This difference in meaning reflects the different fea- 
tures of the beliefs and the goals of an agent. The declarative goals are best 
thought of as achievement goals in this paper. That is, these goals describe a 
goal state that the agent desires to reach. Mainly due to the temporal features 
of such goals many properties of beliefs fail for goals. For example, the fact that 
an agent has the goal to be at home and the goal to be at the movies does not 
allow the conclusion that this agent also has the conjunctive goal to be at home 
and at the movies at the same time. As a consequence, less stringent consistency 
requirements are imposed on goals than on beliefs. An agent may have the goal 
to be at home and the goal to be at the movies simultaneously; assuming these 
two goals cannot consistently be achieved at the same time does not mean that 
an agent cannot have adopted both in the language GOAL. 

In this paper, we assume that the language C used for representing beliefs 
and goals is a simple prepositional language. As a consequence, we do not 
discuss the use of variables nor parameter mechanisms. Our motivation for 
this assumption is the fact that we want to present our main ideas in their 
simplest form and do not want to clutter the definitions below with details. Also, 
more research is needed to extend the programming language with a parameter 
passing mechanism, and to extend the programming logic for GOAL with first 
order features. 

The language C for representing beliefs and goals is extended to a new lan- 
guage Cm which enables us to formulate conditions on the mental state of an 
agent. The language Cm consists of so called mental state formulas. A mental 
state formula is a boolean combination of the basic mental state formulas B(f), 
which expresses that (j> is believed to be the case, and G^, which expresses that 
is a goal of the agent. 

Definition 2.2 (mental state form,ula) 

The set of mental state formulas Cm is defined by: 
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• if G then Bcj) G Cm, 



• if G £, then Gcf) G Cm, 

• if , (^2 G Cm , then -^(pi,ipi /\ (p2 & Cm ■ 

The usual abbreviations for the propositional operators V, — >, and <-> are 
used. We write true as an abbreviation for B{p V -^p) for some p and false for 
-■true. 

The semantics of behef conditions Bcj), goal conditions and mental state 
formulas is defined in terms of the classical consequence relation \=c- 

Definition 2.3 (semantics of mental state formulas) 
Let (S,r} be a mental state. 

• (s,r) Hm B^iff e 

. (s,r) Hm G^AiffTAGr, 

• (i;,r) |=M -"^ iff (s,r} i^^m <(5, 

• (i;,r) |=M A(p2 iff (s,r} |=m <y5i and (i;,r) \=m <p2- 

We write ^5 for the fact that mental state formula ip is true in all mental 
states (S,r}. 

A number of properties of the belief and goal modalities and the relation 
between these operators are listed in Tables |l] and ||. Here, he denotes deriv- 
ability in classical logic, whereas \-m refers to derivability in the language of 
mental state formulas Cm ■ 

The first rule (i?l) below states that mental state formulas that 'have the 
form of a classical tautology' (like {Bip V -^Bip) and G(j)i (B02 — ^ 
are also derivable in \-m- By the necessitation rule (R2), an agent believes all 
classical tautologies. Then, (^1) expresses that the belief modality distributes 
over implication. This implies that the beliefs of an agent are closed under logical 
consequence. Finally, A2 states that the beliefs of an agent are consistent. In 
essence, the belief operator thus satisfies the properties of the system KD (see 
[H H^). Although in its current presentation, our language does not allow 
for nested (belief-) operators, from [^l]. Section 1.7] we conclude that we may 
assume as if om agent has positive {B(p BBcf)) and negative (-iB^ — > B-iBc/)) 
introspective properties: every formula in the system KD45 (which is KD 
together with the two mentioned properties) is equivalent to a formula without 
nestings of operators. 

Axiom A4: below, is a consequence of the constraint on mental states and 
expresses that if an agent believes it does not have a goal to achieve (p. As a 
consequence, an agent cannot have a goal to achieve a tautology: -iGtrue. An 
agent also does not have inconsistent goals (^3), that is, -iGfalse is an axiom 
(see Table H). Finally, the conditions that allow to conclude that the agent has 
a (sub)goal ip are that the agent has a goal ip that logically entails ip and that 
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Rl if 93 is an instantiation of a classical tautology, then Km <y3 

R2 he B(/), for (/) G £ 

Al Km B(<?!>^ V) ^ (B0^ BV') 

A2 (-M -'Bfalse 



Table 1: Properties of Beliefs 



the agent does not believe that is the case. Axiom ^5 below then allows to 
conclude that holds. From now on, for any mental state formula tp, \~m V 
means that that there is a derivation of ip using the proof rules Rl and R2 

and the axioms Al A6. If A is a set of mental state formulas from Cm, 

then A\-M ^ means that there is a derivation of ip using the rules and axioms 
mentioned, and the formulas of A as premises. 



^3 Km ^Gfalse 

A4 Km B(t) -nG(f> 

A5 he ^ ^ -B^A ^ (G0 ^ GV') 



Table 2: Properties of Goals 

The goal modality is a weak logical operator. For example, the goal modality 
does not distribute over implication. A counter example is provided by the goal 
base that is generated from {p,p — > q}. The consequences of goals are only 
computed locally, from individual goals. But even from the goal base {p A 
(p — i- q) one cannot conclude that g is a goal, since this conclusion is blocked 
in a mental state in which q is already believed. Deriving only consequences of 
goals locally ensures that from the fact that G(j> and Gip hold, it is not possible 
to conclude that G(0AV'). This reflects the fact that individual goals cannot be 
added to a single bigger goal; recall that two individual goals may be inconsistent 
(G(/> A G-icj) is satisfiable) in which case taking the conjunction would lead to 
an inconsistent goal. In sum, most of the usual problems that many logical 
operators for motivational attitudes suffer from do not apply to our G operator 



(cf. also |£2|). On the other hand, the last property of Lemma 2.4 justifies to 
call G a logical, and not just a syntactical operator: 



Lemma 2.4 

• G(0 ^ ^) ^ (G0 ^ GV-), 

• G(0 A ((^ ^)) GV-, 

• (G^AG^A) ^ G{cbA^) 

• he iv^i') ^ \=M {Gip ^ G-ip) 
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One finds a similar quest for such weak operators in awareness logics for 
doxastic and epistemic modalities, see e.g. ||, 0. As agents do not want all 
the side-effects of their goals, being limited reasoners they also do not always 
adopt all the logical consequences of their belief or knowledge. However, the 
question remains whether modal logic is the formal tool to reason with and 
about goals. Allowing explicitly for mutually inconsistent goals, our treatment 
of goals resides in the landscape of paraconsistent logic (cf. ||2^). One might 
even go a step further and explore to use linear logic to reason about 

goals, enabling to have the same goal more than once, and to model process 
and resource use in a fine-tuned way. We will not pursue the different options 
for logics of goals in this paper. 

Theorem 2.5 (Soundness and Completeness o/I-m) 
For any ip e Cmi we have 

Proof. We leave it for the reader to check soundness (i.e., the '=>'-direction). 
Here, A\ and A2 are immediate consequences of the definition of a belief as 
a consequ ence from a given consistent set, Ai follows from condition (ii) of 



Definition AA from property (i) and A5 from (Hi) of that same definition. 
For completeness, assume that I/m f. Then -lip is consistent, and we will 
construct a mental state (S, F) that verifies tp. First, we build a maximal h^- 
consistent set A with ^ip £ A. This A can be split in a set S and a set F as 
follows: T, = {(j) \ B(j) £ A} and F = {0 | G(/) G A}. We now prove two properties 
of (S,F): 

1. (S, F) is a mental state 

2. (E, F) satisfies the following coincidence property: 

for aU X e : (S, F) X ^ X e A 

The proofs for these claims are as follows: 



1. We must show that (E, F) satisfies the properties of Definition 2.1. Obvi- 
ously, S is classically consistent, since otherwise we would have B± in the 
hM-consistent set A, which is prohibited by axiom A2. Also, by axiom 
j43, no 7 e F is equivalent to ±. We now show that no 7 e F is classically 
entailed by E. Suppose that we would have that (7i,...,(t„ hp 7, for 
certain cti, . . . , (t„ G S and G G F. Then, by construction of S and F, the 
formulas Bpi, . . . , B(^„, G7 all are members of the maximal hM-consistent 
set A. Since tri, . . . , cr„ he 7, by the deduction theorem for h^, R2 and 
A2 we conclude h^ (Bai, . . . , B(t„) B7. But this means that both B7 
and G7 are members of A, which is prohibited by axiom AA. Finally, we 



show (m) of Definition 2.1, Suppose 7 G F, \=c 7—^7' and that 7' is 
consistent, and not classically entailed by E. We have to 7' G F, and this 
is immediately guaranteed by axiom A5. 
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2. The base case for the second claim is about B<j> and G<j>, with (p E C. We 
have (I],r) iff S iff, by definition of E, {ct | Ba G A} </>• 

Using compactness and the deduction theorem for classical logic, we find 
h(7 (7i A • • • A (Tn) (f), for some propositional formulas (Ti, . . . . (t„. By 
the rule R2 we conclude Km B(cri A ••• A cr„) ^ </>). By Al, this is 
equivalent to Km (Bcti A • • • A B(t„) — > B(^ and, since all the 6(7,(1 < n) 
are members of A, wc have B(/) G A. For the other base case, consider 
(E, r) \=M G(/>, which, using the truth-definition for G, holds iff 7 G F. By 
definition of F, this means that G7 G A, which was to be proven. The 
cases for negation and conjunction follow immediately from this. Hence, 
in particular, we have (E,F) \=m ~"P, and thus 'P- 

2.2 GOAL Agents 

A third basic concept in GOAL is that of an agent capability. The capabilities 
of an agent consist of a set of so called basic actions. The effects of executing 
such a basic action are reflected in the beliefs of the agent and therefore a basic 
action is taken to be a belief update on the agent's beliefs. A basic action thus is 
a mental state tran.sform,er. Two examples of agent capabilities are the actions 
\ns{(f>) for inserting (p in the belief base and del(0) for removing (p from the belief 
base. Agent capabilities directly affect the belief base of the agent and not its 
goals, but because of the constraints on mental states they may as a side effect 
modify the current goals. For the purpose of modifying the goals of the agent, 
two special actions adopt((/)) and drop((/)) are introduced to respectively adopt 
a new goal or drop some old goals. We write Bcap and use it to denote the 
set of all belief update capabilities of an agent. Bcap thus does not include 
the two special actions for goal updating adopt(0) and drop(0). The set of 
all capabihties is then defined as Cap = Bcap U {adopt(^), drop((/)) | <j) G £}. 
Individual capabilities are denoted by a. 

The set of basic actions or capabilities associated with an agent determines 
what an agent is able to do. It does not specify when such a capability should 
be exercised and when performing a basic action is to the agent's advantage. To 
speciiy such conditions, the notion of a conditional action is introduced. A con- 
ditional action consists of a mental state condition expressed by a mental state 
formula and a basic action. The mental state condition of a conditional action 
states the conditions that must hold for the action to be selected. Conditional 
actions are denoted by the symbol b throughout this paper. 

Definition 2.6 (conditional action) 

A conditional action is a pair (p — > do{a) such that (p G Cm and a G Cap. 

Informally, a conditional action (p — > do (a) means that if the mental con- 
dition (fi holds, then the agent may consider doing basic action a. Of course, 
if the mental state condition holds in the current state, the action a can only 
be successfully executed if the action is enabled, that is, only if its precondition 
holds. 
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A GOAL agent consists of a specification of an initial mental state and a set 
of conditional actions. 

Definition 2.7 (GOAL agent) 

A GOAL agent is a triple (11, Sq, Tq) where 11 is a non-empty set of conditional 
actions, and (EoiTo) is the initial mental state. 

2.3 The Operational Semantics of GOAL 

One of the key ideas in the semantics of GOAL is to incorporate into the seman- 
tics a particular commitment strategy (cf. | |26| , |^). The semantics is based on a 
particularly simple and transparent commitment strategy, called blind commit- 
ment. An agent that acts according to a blind commitment strategy drops a 
goal if and only if it believes that that goal has been achieved. By incorporating 
this commitment strategy into the semantics of GOAL, a default commitment 
strategy is built into agents. It is, however, only a default strategy and a pro- 
grammer can overwrite this default strategy by means of the drop action. It 
is not possible, however, to adopt a goal <j) in case the agent believes that <j> is 
already achieved. 

The semantics of action execution should now be defined in conformance with 
this basic commitment principle. Recall that the basic capabilities of an agent 
were interpreted as belief updates. Because of the default commitment strategy, 
there is a relation between beliefs and goals, however, and we should extend the 
belief update associated with a capability to a mental state transformer that 
updates beliefs as well as goals according to the blind commitment strategy. 
To get started, we thus assume that some specification of the belief update 
semantics of all capabilities - except for the two special actions adopt and drop 
which only update goals - is given. Our task is, then, to construct a mental 
state transformer semantics from this specification for each action. That is, we 
must specify how a basic action updates the complete current mental state of 
an agent starting with a specification of the belief update associated with the 
capability only. 

From the default blind commitment strategy, we conclude that if a basic 
action a - different from an adopt or drop action - is executed, then a goal is 
dropped only if the agent believes that the goal has been accomplished after 
doing a. The revision of goals thus is based on the beliefs of the agent. The 
beliefs of an agent represent all the information that is available to an agent to 
decide whether or not to drop or adopt a goal. So, in case the agent believes 
that a goal has been achieved by performing some action, then this goal must be 
removed from the current goals of the agent. Besides the default commitment 
strategy, only the two special actions adopt and drop can result in a change to 
the goal base. 

The initial specification of the belief updates associated with the capabilities 
Bcap is formally represented by a partial function T of type : Bcap x p{C) 
p(£). T(a, S) returns the result of updating belief base E by performing action 
a. The fact that T is a partial function represents the fact that an action may 
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not be enabled or executable in some belief states. The mental state transformer 
function Ai is derived from the semantic function T and also is a partial func- 
tion. As explained, Ai{a, (5],r)) removes any goals from the goal base F that 
have been achieved by doing a. The function A4 also defines the semantics of 
the two special actions adopt and drop. An adopt((/>) action adds (p to the goal 
base if (f) is consistent and ^ is not believed to be the case. A drop((/)) action 
removes every goal that entails <f> from the goal base. As an example, consider 
the two extreme cases: drop(false) removes no goals, whereas drop(true) removes 
all current goals. 

Definition 2.8 (mental state transformer A4) 

Let (S, r) be a mental state, and T be a partial function that associates belief 
updates with agent capabilities. Then the partial function M is defined by: 

A<(a,(E,r)) = (T(a,I]),r\{VGr I T(a,S) V}) 

for a G Bcap, if T(a, S) is defined 

A4{a, (S, r)) is undefined for a G Bcap if T(a, S) is undefined 

Ai(drop(,^),(E,r)) = (i],r\{VGr I vNc./-}) 
Ai(adopt(.^),(s,r)) = (i],ru{,^'|s^M0'& \=c<l>^ <!>'}) 

if ~^4> and S 4> 
A1(adopt((;6), (S,r)) is undefined if S \=c (/) or \=c -x/) 

The semantic function Ad maps an agent capability and a mental state to a 

new mental state. The capabilities of an agent are thus interpreted as mental 
state transformers by A4. Although it is not allowed to adopt a goal (p that is 
inconsistent - an adopt(false) is not enabled - there is no check on the global 
consistency of the goal base of an agent built into the semantics. This means 
that it is allowed to adopt a new goal which is inconsistent with another goal 
present in the goal base. For example, if the current goal base F contains p, it 
is legal to execute the action adopt(-ip) resulting in a new goal base containing 
p,^p, (if ^p was not already believed). Although inconsistent goals cannot 
be achieved at the same time, they may be achieved in some temporal order. 
Individual goals in the goal base, however, are required to be consistent. Thus, 
whereas local consistency is required (i.e. individual goals must be consistent), 
global consistency of the goal base is not required. 

The second idea incorporated into the semantics concerns the selection of 
conditional actions. A conditional action ip do(a) may specify conditions on 
the beliefs as well as conditions on the goals of an agent. As is usual, conditions 
on the beliefs are taken as a precondition for action execution: only if the agent's 
current beliefs entail the belief conditions associated with (f the agent will select 
a for execution. The goal condition, however, is used in a different way. It is 
used as a means for the agent to determine whether or not the action will help 
bring about a particular goal of the agent. In short, the goal condition specifies 
where the action is good for. This does not mean that the action necessarily 
establishes the goal immediately, but rather may be taken as an indication that 
the action is helpful in bringing about a particular state of affairs. 

In the definition below, we assume that the action component 11 of an agent 
(n, So,Fo) is fixed. The execution of an action gives rise to a computation 
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step formally denoted by the transition relation — > where h is the conditional 
action executed in the computation step. More than one computation step 
may be possible in a current state and the step relation — > thus denotes a 
possible computation step in a state. A computation step updates the current 
state and yields the next state of the computation. Note that because is a 
partial function, a conditional action can only be successfully executed if both 
the condition is satisfied and the basic action is enabled. 



Definition 2.9 (action selection) 
Let (S, r) be a mental state and h 
If 



= Lp (io(a) G n. Then, as a rule, we have: 



• the mental condition (p holds in (S,r), i.e. (S,r) ^ and 

• a is enabled in (S,r}, i.e. A^(a, (I],r}) is defined, 

then (S,r) — A^(a, (I],r)) is a possible computation step. The relation — > 
is the smallest relation closed under this rule. 

Now, the semantics of GOAL agents is derived directly from the operational 
semantics and the computation step relation — > . The meaning of a GOAL 
agent consists of a set of so called traces. A trace is an infinite computation 
sequence of consecutive mental states interleaved with the actions that are sched- 
uled for execution in each of those mental states. The fact that a conditional 
action is scheduled for execution in a trace does not mean that it is also enabled 
in the particular state for which it has been scheduled. In case an action is 
scheduled but not enabled, the action is simply skipped and the resulting state 
is the same as the state before. In other words, enabledness is not a criterion 
for selection, but rather it decides whether something is happening in a state, 
once selected. 

Definition 2.10 (trace) 

A trace s is an infinite sequence so , &o 7 si , &i , S2 , • ■ • such that Si is a mental 

state, hi is a conditional action, and for every i we have: Si Si+i, or hi is 
not enabled in Si and Si ~ s^+i. 

An important assumption in the semantics for GOAL is a fairness assump- 
tion. Fairness assumptions concern the fair selection of actions during the exe- 
cution of a program. In our case, we make a weak fairness assumption [po[ . A 
trace is weakly fair if it is not the case that an action is always enabled from 
some point in time on but is never selected for execution. This weak fairness 
assumption is built into the semantics by imposing a constraint on traces. By 
definition, a fair trace is a trace in which each of the actions is scheduled in- 
finitely often. In a fair trace, there always will be a future time point at which 
an action is scheduled (considered for execution) and by this scheduling policy 
a fair trace implements the weak fairness assumption. However, note that the 
fact that an action is scheduled does not mean that the action also is enabled 
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(and therefore, the selection of the action may result in an idle step which does 
not change the state). 

The meaning of a GOAL agent now is defined as the set of fair traces in 
which the initial state is the initial mental state of the agent and each of the 
steps in the trace corresponds to the execution of a conditional action or an idle 
transition. 

Definition 2.11 (meaning of a GOAL agent) 

The meaning of a GOAL agent (11, Eo, Fq) is the set of fair traces S such that 
for s e 5 we have sq = (S^OiTo). 

2.4 Mental States and Enabledness 

We formally said that a capability a G Cap is enabled in a mental state (5],r) 
in case A^(a,(E,r)) is defined. This definition implies that a belief update 
capability a S Bcap is enabled if T(a, S) is defined. Let us assume that this only 
depends on the action a -this seems reasonable, since a paradigm like AGM ([|l|) 
only requires that a revision with fails iff Lp is classically inconsistent, whereas 
expansions and contractions succeed for all ip, hence the question whether such 
an operation is enabled does not depend on the current beliefs. A conditional 

action h is enabled in a mental state (S, F) if there are S', F' such that (E, F) — ^ 
(S'jF'). Note that if a capability a is not enabled, a conditional action p 
do{a) is also not enabled. The special predicate enabled is introduced to denote 
that a capability a or conditional action b is enabled (denoted by enabled{a) 
respectively enabled{b)). 

The relation between the enabledness of capabilities and conditional actions 
is stated in the next table together with the fact that drop((/>) is always enabled 
and a proof rule for deriving enabled{adopt{4>)) . Let Cme be the language 
obtained by Boolean combinations of mental state formulas and enabledness 
formulas. We denote derivability in the system for this language by ^ me- Then, 
'tme consists of the axioms and rules for h^, plus 



El enabled{ip do{a)) ^ (ip A enabled{a)), 
E2 enabled {drop{(f))), 

-R3 ^(f' ^^ME ^ enabled {adiopt{(l)) 

R4: \=c ^^ME -^enabled{adopt{(f))) 

^ME enabledia) if T(a, •) is defined (a e Bcap) 



Table 3: Enabledness 

Rule i?5 enforces that we better write given a belief revision function 
T, but in the sequel we will suppress this T. The semantics \^me for Cme is 
based on truth in pairs (S, F), T, where (S, F) is a mental state and T a partial 
function for belief updates. For formulas of the format B(p and G(/9, we just use 



the mental state and Definition 2.3 to determine their truth. For enabledness 
formulas, we have the following: 
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Definition 2.12 (Truth of enabledness) 

• (S,r),T |=ME enabled{a) iff T(a,E;) is defined 

• (S,r),T \=ME enabled{drop{(p)) iff true 

• (E,r),T |=ME enabled{adopt{(f>)) iff -^<P and {i:,T),T \=me 

• (S,r),T \=ME enabled{ip — »■ rfo(a)) iff (S,r),T |=m£; (!> and at the same 
time (S,r),T |=mb enabled{a) 

Note that we can summarize this definition to: 

• (S,r),T |=ME enabled{a) iff A^(a, (S,r)) is defined for a e Cap, 

• {T,,T),T \=ME enabled{b) iff \=me <f and there are S',r' such that 
(S,r) — ^ (S',r') for conditional actions where b = ip ^ do{a). 

Theorem 2.13 (Soundness and Completeness of \-me) 
We have, for all formulas ip in £me, 

\-ME V iff \=ME V 

Proof. Again, checking soundness is straightforward and left to the reader. 
For the converse, we have to make a complexity measure explicit for £me- 
formulas, along which the induction can proceed. It suffices to stipulate that 
the complexity of enabled{ip — > do (a) is greater than that of B'0 and enabled{a). 
Furthermore, the complexity of enabled{a(iopt{(j)) is greater than that of (-i)B^. 
Now, suppose that I/mb f, i-<2., ^ip is consistent. Note that the language £me 
is countable, so that we can by enumeration, extend {^ip} to a maximal \-me- 
consistent set A. From this A, we distill a pair (E,r),T as follows: T, = {(p \ 
Bip G A}, r — {ip \ G(p G A}, and T(a, S) is defined iff enabled{a) G A, for any 
belief capability a. We claim, for all % e Cme- 

(s,r),ThxiffxGA 

For formulas of type B'lp and Gtp this is easily seen. Let us check it for 
enabledness formulas. 

• X = enabled{a), with a a belief capability. By construction of T, the result 
immediately holds 

• X = enabled{drop{(f>)) . By construction of A, every enabled{drop{(f>)) is an 
element of A (because of axiom E2), and also, every such formula is true 

in (E,r),r. 

• X = enabled{adopt{(j))) . Suppose (E,r),T \=me X- Then, ^ -k/) and 
(S,r),T Y=ME B(/). By the induction hypothesis, we have that B><f> ^ A, 
hence -iB^ e A, and, by ii3, ena6/ed(adopt^)) e A. For the converse. 
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suppose e7ia6/erf(adopt((/))) G A. Then (by i?4), wc cannot have that 
\=c ~^<t>- Hence, and by ii3, we also have -iB^ G A and hence, 

by applying the induction hypothesis, (E,r),T |=c -iB^. Since i23 is a 
sound rule, we finally conclude that (S,r),T \=me ena6/ed(adopt(^)). 

• X = enabled{ijj a). We can write this as V A enabled{a) and then use 
the induction hypothesis. 

3 A Personal Assistant Example 

In this section, we give an example to show how the programming language 
GOAL can be used to program agents. The example concerns a shopping agent 
that is able to buy books on the Internet on behalf of the user. The example 
provides for a simple illustration of how the programming language works. The 
agent in our example uses a standard procedure for buying a book. It first goes 
to a bookstore, in our case Am.com. At the web site of Am.com it searches for 
a particular book, and if the relevant page with the book details shows up, the 
agent puts the book in its shopping cart. In case the shopping cart of the agent 
contains some items, it is allowed to buy the items on behalf of the user. The 
idea is that the agent adopts a goal to buy a book if the user instructs it to do 
so. 

The set of capabilities Bcap of the agent is defined by 

{goto-website{site) , search{book) , put-in-shopping-cart{book) , pay-cart} 

The capability goto-website{site) goes to the selected web page site. In our 
example, relevant web pages are the home page of the user, the main page of 
Am.com, web pages with information about books to buy, and a web page 
that shows the current items in the shopping cart of the agent. The ca- 
pability search{hook) is an action that can be selected at the main page of 
Am.com and selects the web page with information about hook. The action 
put-in-shopping-cart(book) can be selected on the page concerning book and 
puts book in the cart; a new web page called ContentCart shows up showing the 
content of the cart. Finally, in case the cart is not empty the action pay-cart 
can be selected to pay for the books in the cart. 

In the program text below, we assume that book is a variable referring to the 
specifics of the book the user wants to buy (in the example, we use variables as 
a means for abbreviation; variables should be thought of as being instantiated 
with the relevant arguments in such a way that predicates with variables reduce 
to propositions). The initial beliefs of the agent are that the current web page is 
the home page of the user, and that it is not possible to be on two different web 
pages at the same time. Wc also assume that the user has provided the agent 
with the goals to buy The Intentional Stance by Daniel Dennett and Intentions, 
Plans, and Practical Reason by Michael Bratman. 
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n = { 

B{current_website{hpage{user)) V current_website{ContentCart)) 

AG{bought{book)) — > do{goto_website{Am.com)), 
B{current_website{Am. com)) A ^B(in_cart{book))A 

Q[bought{book)) — > do{search{book)), 
B{current_website{book)) A Q[bought{book)) 
—t do [put_in_shopping_cart ( book ) ) , 
B{in_cart{book)) A G {bought (book)) —^ do{pay_cart)}, 
So = {current_webpage{hpage{user)), 

Vs,s'((s ^ s' A current _'webpage{s)) — > -icurrent^webpage^s'))}, 
Tq — {boughtlThe Intentional Stance) 

A ftou^/if (Intentions, Plans and Practical Reason)} 
GOAL Shopping Agent 

Some of the details of this program will be discussed in the sequel, when we 
prove some properties of the program. The agent basically follows the recipe for 
buying a book outlined above. For now, however, just note that the program 
is quite flexible, even though the agent more or less executes a fixed recipe for 
buying a book. The flexibility results from the agent's knowledge state and 
the non-determinism of the program. In particular, the ordering in which the 
actions are performed by the agent - which book to find first, buy a book one 
at a time or both in the same shopping cart, etc. is not determined by the 
program. The scheduling of these actions thus is not fixed by the program, and 
might be fixed arbitrarily on a particular agent architecture used to run the 
program. 



4 Logic for GOAL 

On top of the language GOAL and its semantics, we now construct a temporal 
logic to prove properties of GOAL agents. The logic is similar to other temporal 
logics but its semantics is derived from the operational semantics for GOAL. 
Moreover, the logic incorporates the belief and goal modalities used in GOAL 
agents. We first informally discuss the use of Hoare triples for the specification 



of actions. In Section 4.3 we give a sound an complete system for such triples. 
These Hoare triples play an important role in the programming logic since it 
can be shown that temporal properties of agents can be proven by means of 



proving Hoare triples for actions only. Finally, in 4.4 the language for express- 
ing temporal properties and its semantics is defined and the fact that certain 
classes of interesting temporal properties can be reduced to properties of actions, 
expressed by Hoare triples, is proven. 



4.1 Hoare Triples 

The specification of basic actions provides the basis for the programming logic, 
and, as we will show below, is all we need to prove properties of agents. Because 
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they play such an important role in the proof theory of GOAL, the specifica- 
tion of the basic agent capabilities requires special care. In the proof theory 
of GOAL, Hoare triples of the form {(p} b {ip}, where (p and ijj are mental 
state formulas, are used to specify actions. The use of Hoare triples in a formal 
treatment of traditional assignments is well- understood Q]. Because the agent 
capabilities of GOAL agents are quite different from assignment actions, how- 
ever, the traditional predicate transformer semantics is not applicable. GOAL 
agent capabilities are mental state transformers and, therefore, we require more 
extensive basic action theories to formally capture the effects of such actions. 
Hoare triples are used to specify the postconditions and the frame conditions of 
actions. The postconditions of an action specify the effects of an action whereas 
the frame conditions specify what is not changed by the action. Axioms for the 
predicate enabled specify the preconditions of actions. 

The formal semantics of a Hoare triple for conditional actions is derived from 
the semantics of a GOAL agent and is defined relative to the set of traces Sa 
associated with the GOAL agent A. A Hoare triple for conditional actions thus 
expresses a property of an agent and not just a property of an action. The 
semantics of the basic capabilities are assumed to be fixed, however, and are 
not defined relative to an agent. 

Definition 4.1 (semantics of Hoare triples for basic actions) 

A Hoare triple for basic capabilities {tp} a {ip} means that for all S],r 

• (E, F) ^ A enabled{a) X(a, (S, F)) \= and 

• (E, F) h (/3 A -nenabled{a) ^ (S, F) |= tp. 

To explain this definition, note that we made a case distinction between 
states in which the basic action is enabled and in which it is not enabled. In 
case the action is enabled, the postcondition ip of the Hoare triple {ip} a {ip} 
should be evaluated in the next state resulting from executing action a. In case 
the action is not enabled, however, the postcondition should be evaluated in the 
same state because a failed attempt to execute action a is interpreted as an idle 
step in which nothing changes. 

Hoare triples for conditional actions are interpreted relative to the set of 
traces associated with the GOAL agent of which the action is a part. Below, we 
write p[si] to denote that a mental state formula p holds in state Si. 

Definition 4.2 (semantics of Hoare triples for conditional actions) 
Given an agent A, a Hoare triple for conditional actions {p} b {ip} (for A) 
means that for all traces s G Sa and i, we have that 

{ip[st] A b = bi e s) ^ ip[si+i] 

where bi E s means that action bi is taken in state i of trace s. 

Of course, there is a relation between the execution of basic actions and that 
of conditional actions, and therefore there also is a relation between the two 
types of Hoare triples. The following lemma makes this relation precise. 
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Lemma 4.3 Let A he a GOAL agent and Sa be the meaning of A. Suppose 
that we have {(p Aip} a {<f'} and Sa |= A -'ip) — + ip'. Then we also have 
{^} ^ ^ do(a) W'}. 

Proof: We need to prove that {ip[si] A {^p ^ do{a)) = bi G s) =^ (p'[si+i]. 
Therefore, assume ip[si] A {tp do{a)) = bi & s). Two cases need to be 
distinguished: The case that the condition tp holds in Si and the case that 
it does not hold in s^. In the former case, because we have {if Aip} a {(p'} we 
then know that s^+i \= (p' . In the latter case, the conditional action is not 
executed and s^+i = ,Sj. From ((p A -^ip) p')[si], ip[si] and -■■^[sj] it then 
follows that (/?'[si_|_i] since (p' is a state formula. 

The definition of Hoare triples presented here formalises a total correctness 
property. A Hoare triple {p>} b {tp} ensures that if initially ip holds, then an 
attempt to execute b results in a successor state and in that state ip holds. This 
is different from partial correctness where no claims about the termination of 
actions and the existence of successor states are made. 

4.2 Basic Action Theories 

A basic action theory specifies the effects of the basic capabilities of an agent. 
It specifies when an action is enabled, it specifies the effects of an action and 
what does not change when an action is executed. Therefore, a basic action 
theory consists of axioms for the predicate enabled for each basic capability, 
Hoare triples that specify the effects of basic capabilities and Hoare triples that 
specify frame axioms associated with these capabilities. Since the belief update 
capabilities of an agent are not fixed by the language GOAL but are user- 
defined, the user should specify the axioms and Hoare triples for belief update 
capabilities. The special actions for goal updating adopt and drop are part of 
GOAL and a set of axioms and Hoare triples for these actions is specified below. 

4.2.1 Actions on beliefs: capabilities of the shopping assistant 

Because in this paper, our concern is not with the specification of basic action 
theories in particular, but with providing a programming framework for agents 
in which such specifications can be plugged in, we only provide some example 
specifications of the capabilities defined in the personal assistant example that 
we need in the proof of correctness below. 

First, we specify a set of axioms for each of our basic actions that state when 
that action is enabled. Below, we abbreviate the book titles of the example, and 
write T for The Intentional Stance and / for Intentions, Plans, and Practical 
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Reason. In the shopping agent example, we then have: 

enabled{goto_website{site)) ^ true, 

enabled{search{book)) ^ B{current_website{Amazon.com,)), 
enabled{put-in_shopping_cart{book)) ^ B{current_website{book)), 
enabled{pay-cart) ^ 
((Bm_cart(r) V Bm_cart(/)) A Bcurrent-website{ContentCart)). 

Second, we list a number of effect axioms that specify the effects of a capa- 
bility in particular situations defined by the preconditions of the Hoare triple. 

• The action goto_website{site) results in moving to the relevant web page: 
{true} goto_website{site) {Bcurr6nt_website{site)}, 

• At Amazon.com, searching for a book results in finding a page with rele- 
vant information about the book: 

{ B current_wehsite{Amazon . com) } search ( book ) { B current-website ( book ) } 

• On the page with information about a particular book, selecting the action 
put_in-shopping-cart{book) results in the book being put in the cart; also, 
a new web page appears on which the contents of the cart are listed: 

{ B currant _website{book)} 

put-in-shopping_cart ( book) 
{B(in_cart{book) A current_website{ContentCart))} 

• In case book is in the cart, and the current web page presents a list of all 
the books in the cart, the action pay-cart may be selected resulting in the 

buying of all listed books: 

{B(in_cart{book) A current_website{ContentCart))} 
pay-cart 

{^Bin-cart{book) A B{bought{book) A current-website{Amazon.com))} 

Finally, we need a number of frame axioms that specify which properties 
are not changed by each of the capabilities of the agent. For example, both the 
capabilities goto_website{site) and search{book) do not change any beliefs about 
iri-cart. Thus we have, e.g.: 

{Bin_cart{book)} goto-website{site) {Bin-cart{book)} 
{Bin-cart{book)} search{book) {Bin-ca'rt{book)} 

It will be clear that we need more frame axioms than these two, and some of 
these will be specified below in the proof of the correctness of the shopping 
agent. 

It is important to realise that the only Hoare triples that need to be specified 
for agent capabilities are Hoare triples that concern the effects upon the beliefs of 
the agent. Changes and persistence of (some) goals due to executing actions can 
be derived with the proof rules and axioms below that are specifically designed 
to reason about the effects of actions on goals. 
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4.2.2 Actions on goals 



A theory of the behef update capabihties and their effects on the behefs of an 
agent must be complemented with a theory about the effects of actions upon the 
goals of an agent. Such a theory should capture both the effects of the default 
commitment strategy as well as give a formal specification of the the drop and 
adopt actions. Only in Section 4.3 we aim at providing a complete system, in 
the discussion in the current section, there are dependencies between the axioms 
and rules discussed. 



Default commitment strategy The default commitment strategy imposes 
a constraint on the persistence of goals. A goal persists if it is not the case 
that after doing a the goal is believed to be achieved. Only action drop((/)) is 
allowed to overrule this constraint. Therefore, in case a / drop((/)), we have that 
{G(j)} a {B(f> V G(j)} (using the rule for conditional actions from Table ||, one can 
derive that this triple also holds for general conditional actions b, rather than 
just actions a). The Hoare triple precisely captures the default commitment 
strategy and states that after executing an action the agent either believes it 
has achieved or it still has the goal if was a goal initially. 



a ^ drop(0) 
{G4>} a {B<^VG(/)} 



Table 4: Persistence of goals 



A similar Hoare triple can be given for the persistence of the absence of a 
goal. Formally, we have 

{-G0} & {-B0V-G0} (1) 

This Hoare triple states that the absence of a goal (p persists, and in case 
it does not persist the agent does not believe (j) (anymore). The adoption of 
a goal may be the result of executing an adopt action, of course. However, it 
may also be the case that an agent believed it achieved (p but after doing b no 
longer believes this to be the case and adopts as a goal again. For example, 
if the goal base is {p A q} and the belief base S — {p}, then the agent does not 
have a goal to achieve p because it already believes p to be the case; however, 
in case an action changes the belief base such that p is no longer is believed, the 
agent has a goal to achieve p (again). This provides for a mechanism similar to 
that of maintenance goals. We do not need the Hoare triple (^ as an axiom, 
however, since it is a direct consequence of the fact that B0 — *■ -iGi/) (this is 
exactly the postcondition of (^. Note that the stronger {^Gcp} b {^G(j)} does 
not hold, even ii b ^ (p ^ do{adopt{(j))). This occurs for example if we have 
G{p A g) A Bp. Then the agent does not have p as a goal, since he believes it 
has already been achieved, but, if he would give up p as a belief, it becomes to 
be a goal. 
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In the semantics of Hoare triples (Definition 4.2) we stipulated that if a is 
not enabled, we verify the postcondition in the same state as the pre-condition: 



-^enahled{a) 
{lp} a {ip} 



Table 5: Infeasible actions 



Frame properties on Beliefs The specification of the special actions drop 
and adopt involves a number of frame axioms and a number of proof rules. 
The frame axioms capture the fact that neither of these actions has any effect 
on the beliefs of an agent. Note that, combining such properties with e.g. the 
Consequence Rule (Table |l^) one can derive the triple {BV'} adopt((^) {-'G?/'} ■ 

{B0} adopt(V') {B0} {^B0} adopt(V') {-B0} 
{Be/)} drop(V') {B0} {^B(/.} drop(V^) {-B0} 



Table 6: Frame Properties on Beliefs for adopt and drop 

(Non-)effects of adopt The proof rules for the actions adopt and drop capture 
the effects on the goals of an agent. For each action, we list proof rules for the 
effect and the persistence ('non-effect') on the goal base for adoption (Table 0) 
and dropping (Table ||) of goals, respectively. 

An agent adopts a new goal </> in case the agent does not believe (j) and </> is 
not a contradiction. Concerning persistence, an adopt action does not remove 
any current goals of the agent. Any existing goals thus persist when adopt is 
executed. The persistence of the absence of goals is somewhat more complicated 
in the case of an adopt action. An adopt((/)) action does not add a new goal ■0 
in case V' is not entailed hy (j) oi ip is believed to be the case: 



Effects of adopt 

{-B0} adopt(<?!)) 
Non-effect of adopt 

{GV-} adopt(0) {G^A} {^G0} adopt(V') {-G0} 



Table 7: (Non-)effects of adopt 

A drop action drop(0) results in the removal of all goals that entail (f). This 
is captured by the first proof rule in Table 
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Effects of drop 

he Tp 



{G^} drop(0) {-GV'} 

Non-Effects of drop 
{^G(/)} drop(V') {^G(0 A A drop(?/') {G(/)} 



Table 8: (Non-)effects of drop 

Concerning persistence of goals under drop: a drop action drop((/)) never 
results in the adoption of new goals. The absence of a goal "0 thus persists when 
a drop action is executed. It is more difficult to formalise the persistence of a 
goal with respect to a drop action. Since a drop action drop((/)) removes goals 
which entail 0, to conclude that a goal ip persists after executing the action, we 
must make sure that the goal does not depend on a goal (is a subgoal) that is 
removed by the drop action. In case the conjunction A ■0 is not a goal, we 
know this for certain. 

The basic action theories for GOAL include a number of proof rules to 
derive Hoare triples. The Rule for Infeasible Actions (Table ||) allows to derive 
frame axioms for an action in case it is not enabled in a particular situation. 
The Rule for Conditional Actions allows the derivation of Hoare triples for 
conditional actions from Hoare triples for capabilities. This rule is justified 



by lemma 4.3. Finally, there are three rules for combining Hoare triples and 
for strengthening the precondition and weakening the postcondition: they are 
displayed in Table |l^. 

Finally, we list how one goes from simple actions a to conditional actions 6, 
and how to use complex formulas in pre- and post-conditions. 



[if hip} z {ip'},{ip ^' 

y} i, ^ do(a) {^'} 



Table 9: Rule for conditional actions 



We did not aim in this section at giving a weakest set of rules: in fact, the 
rules that manipulate the pre- and post-conditions in Table |l^ for conditional 
actions h could already be derived if we had only given them from simple actions 
a. 

4.3 A Complete Hoare System 

We now address the issue of finding a complete Hoare system for GOAL. Let 
{p} S {cr} denote that the Hoare triple with pre-condition p and postcondition a 
is derivable in the calculus R that we are about to introduce, and let denote 
the truth of such assertions, that is, |=// determines the truth of mental state 
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Consequence Rule: Conjunction Rule: 

-> t/g^jy} a V'' h {il}i},{ip2} b {-02} 

Disjunction Rule: 

{</5l V (^2} & {0} 



Table 10: Structural rules 



formulas (Definition 2.3), that of formulas with enabled{—) (Definition 2.12) and 



that of Hoare triples (Definition 4.1), on mental states (S,r). From now on, the 
statement S ranges over basic actions a and conditional actions b. Then, this 
subsection wants to settle whether our calculus H is sound and complete, i.e. 
whether it can be proven that, for any pre- and postcondition p and a € Cm, 

{p} S {a} ^ hff {P} S {a} 

Finding such a complete system is, even for 'ordinary deterministic pro- 
grams', impossible, since such programs are interpreted over domains that in- 
clude the integers, and by Godel's Incompleteness Theorem, we know that ax- 
iomatize a domain that includes those integers (cf. [^). Here, our domain is not 
that of the integers, but, instead, we will assume a completeness result for the 
basic capabilities Bcap that modify the belief base, so that we can concentrate 
on the actions that modify the goals. 

Definition 4.4 (General Substitutions) 

We define the following general substitution scheme. Let a, a, (3 be mental 
state formulas, and X a variable ranging over formulas from C. Then (3{X) 
denotes that X may occur in /3. Let C{X) denote a condition on X. Then 
a[a/(3{X) I C(X)] denotes the formula that is obtained from a by substituting 
all occurrences in a of /3{X) for which C{X) holds, by a. 

For instance, the result of {Gp A ^Gq A Gs)[Br/G(f) \ {p A q) ^ <j)] is 
Br A -.Br A Gs. 

Definition 4.5 (The system H) 

The valid Hoare triples for GOAL are as follows: 
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Belief Capabilities {/o(a)} a {cr{a)} 

Adopt {{enabled{adopt{(t))) A a[^B<j)' / Gcf)' \hc <j) </>']) 

V {-^enahled{adopt{(j))) A CT)}{adopt(0)}{CT} 
Drop {a[true/^Q(j)' \ ^c (t>' ^ ^rop{(t>) {(^} 

{p A V} a {a), \=ME {o- A -^ip) p 



Conditional Actions {p} ijj — > do{a) {a} 

hAiii f7 — ^, {^} a {t.;}, =:ui. — fT 



Consequence Rule {p} a {tr} 

Lemma 4.6 (Substitution Lemma) 

(i) Let (S,r) e?zaWed(adopt((/))) and let (S',r') = A^(adopt(0), (E, T)). 
Then: 

(E, r) ahB07G</>' |hc </> ^ 0'] ^ (E', L') a 

(m) Let (S',r') =>l(drop(^!>),(S,r)). Then: 

(E,r) \=M a[twe/^G<l>' \ \- c <f>' ^ <f>] ^ (E',r') a 

Proof. We only prove (i), the proof of (ii) is similar. We prove (i) by induction 
on the mental state formula a. 

1. a is of the form Gx. We distinguish two cases. 

(a) ^ X- By definition of A4(adopt(0), (E, F)), we immediately see 
that (S,r) \=M -Bx iff (S',r') \=M Gx- 

(b) \/c ([> ^ X- The definition of A^(adopt((/)), (E, F)) guarantees that 
the adopt has no effect on the fact that x is a goal, thus we have 
(E,F) \=H Gxiff A^(adopt(0), (E,F)) \=h Gx- Also, the substitution 
has no change as an effect: (Gx)[~'B0'/G0' \\-c X ^ = ^x, and 
we have the desired result. 

2. a if of the form Bx- In this case, the substitution had no effect, and also, 
since the adopt has no effect on the belief base, we have that (S, F) \=m Bx 

iff (s',r') Bx. 

3. The cases that a is a negation or a conjunction of mental states, follows 
immediately. 

Lemma 4.7 (Soundness of \-h) 

for any pre- and postcondition p and a S Cm, {p} S {a} => \=h {p} S {a} 
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Proof. Soundness of Belief Capabilities is assumed. The cases for adopt and 



drop immediately follow from Lemma 4.6. The soundness of Consequence Rule 



is easily proven using Theorem p.5[ . Let us finally consider the rule Condi- 
tional Actions. Suppose that \=h {a Atp} 3 {p} (1), and \=me {p A ^V) ~^ 
(2), and take an arbitrary mental state (5],r). We have to demonstrate that 
(S,r) \=ME {p} V' do{a) {a} (3). Hence, we assume that (S,r) \=me P- We 
then distinguish two cases. First, assume (S,r) \^er V-"- Then, by our assump- 
tion (1), we have that (E',r') |=m cr, for (S',r') = A^(a, (S,r)). The second 
case is the one in which (S,r) ^, i-e., (S,r} |=m By (2), we know 

that (S]',r') hfifl for any (S',r') for which (S,r) (S',r'). All in all, we 
have proven (3). 

We first introduce the notion of weakest liberal precondition, originally due 
to Dijkstra (0). However, we introduce it immediately in the syntax. 

Definition 4.8 (Weakest Liberal Precondition) 

For S = a', adopt(0)drop((/)) and ?/> — ^ a, (a' a belief capability, a any ba- 
sic capability), we define the weakest liberal precondition for S to achieve cr, 
wlp{S,a) e Cm, as follows: 

1. If a' is a belief capability, and \-h {CT^^(a')} a' {a{a')} is the rule for a', 
then wlp{a' ,a{a')) = a^^ 

2. wlp{adopt{(l)),a) = {enabled{adopt{(f))) A a[-nB(l)' /Gcf)' \\-c (/> -> (f)') 
V (-iena6/ed(adopt((/))) A a) 

3. wlp{drop{(j)),a) = a[true/^G(j)' | he 

4. wlp(tp — > a, (t) = (-0 A wlp{a, a)) V (-i?/; A u) 

Note that the weakest precondition wlp{S , a) is indeed an mental state for- 
mula. 

Lemma 4.9 (Weakest Precondition Lemma) 

We have: {wlp{S, a} S {cr}, for every 5" and every postcondition a. 
Proof. For S = a', adopt((/)), drop(0), this follows immediately from Defini- 



tion 4.5. For conditional actions 6 = ■(/;—> a, we have to prove 



\-H {(■0 A wlp{a, cr)) V {^t/j A cr)} -0 — > a {a} 

Let us abbreviate {ip A wlp{a, cr)) V (-1-0 A a) to cr^^"'^. The induction hypothesis 
tells us that \-h {wlp{a,a)} a {cr} (1). Note that \=m cr^"^ wlp{a,a). Hence, 
by the Consequence Rule and (1), we have \-h {cr^^} a {cr} (2). Obviously, we 
also have (cfT^ A -iV-") — > cr (3). Applying the Conditional Actions rule to 
(2) and (3), we conclude \-h {f^^} ^ a {cr}, which was to be proven. 

Lemma 4.10 

\=H {p} S {cr} ^\=M P wlp{S, cr) 
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Proof. We even prove a stronger statement, i.e., that for all mental states 
(E,r}, if (S,r) hff {p} S {a}, then (S,r) P ^ 'wlp{S,a). To prove 

this, we take an arbitrary (S,r) for which both (S,r) |=m P and (S,r) 
{p} 5' {a} and we then have to show that (S,r) wlp{S,a). 

I. S ^ adopt(0). We know that (S,r) \=h {p} adopt(0) {a}. We distinguish 
two cases, the first of which says that (S,r) \=m -^enabled{adopt{(j))). 
Then we have a transition from (S,r) to itself, and hence (S,r) \=m 
(T, and hence (S,r) ^enabled{adopt{(f)j) A a, so that (S,r) \=m 

wZp(adopt(0), (t). In the sec ond case, (S,r} \=m enabled{adopt{(f))) . By 
the Substitution Lemma 4.6, case («), we then immediately see (S,r) 



a[^B4>' / G(j)' \\-c <i> and hence (S,r) |=m wlp{adopt{(f), a). 

2. S = drop((/)). This case follows immediately from the Substitution Lemma 
and the definition of wlp{drop{(t), a). 

3. S = 'ijj^a. We know that (E, T) \=h {p} ip ^ a {a} and that (S, T) \=m 
p. If (S, r) ~'fp, then the transition belonging to S ends up in (E, F), 
and hence we then have (E,r} \=m A it, and in particular (E,r) 
'wlp{S , a). In the other case we have (E, V) \=m V^j and we then know that 
(S,r) \=H {p} a {cr}. By induction we conclude that (E,r) \=m wlp{a,a, 
and hence (E,r) |=m wlp{tp a),t7). 



Theorem 4.11 (Completeness of hi^)) 

For any pre- and postcondition p and a G Cm , 

{p} S {a} ^ hff {P} S {a} 



Proof. Suppose {p} S {a}. The Weakest Precondition Lemma (4.9) 
tells us that \-h {'wlp{S,a)} S {a} (1). By Lemma 4.10, we have \=m cr 



wlp{S, a) (2). We finally apply the Consequence Rule to (1) and (2) to conclude 
to conclude that {c} S {p}. 



4.4 Temporal logic 

On top of the Hoare triples for specifying actions, a temporal logic is used 
to specify and verify properties of GOAL agents. Two new operators are in- 
troduced. The proposition init states that the agent is at the beginning of 
execution and nothing has happened yet. The second operator until is a weak 
until operator, ip until ip means that ■0 eventually becomes true and ip is true 
until becomes true, or -0 never becomes true and ip remains true forever. 

Definition 4.12 (language of temporal logic Ct based on C) 
The temporal logic language Ct is inductively defined by: 

• init G Ct, 

• enabled{a), enabled{ip — > do{a)) G Ct for a G Cap, 
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• if (j) G C, then B^, G(j) e Ct, 

• if e £t, then -11^, y> A V' € £t, 

• if (fi,tl) & jCt, then ip until tjj G jCt- 

A number of other well known temporal operators can be defined in terms of 
the operator until . The always operator Oip is an abbreviation for ip until false, 
and the eventuality operator oip is defined as -iD-k^ as usual. 

Temporal formulas are evaluated with respect to a trace s and a time point 
i. State formulas like B^, G^', enabled{a) etc. are evaluated with respect to 
mental states. 

Definition 4.13 (semantics of tem,poral formulas) 
Let s be a trace and i be a natural number. 

• s,i\^ init iff i = 0, 

• s,i\= enabled{a) iff enabled {a)[si], 

• s,i \= enabled{ip rfo(a)) iff enabled{(p — > do{a))[si], 

• s,i\=B(j)iff B(j)[si], 

• s, i ^ iff G(ji[si], 

• s,i \= iff s,i ^ (fi, 

• s,i \= Alp iS s,i \= (fi and s,i \= ip, 

• s,i until Ip iff 3j > i{s,j \= ip A V k{i < k < j{s,k j= (p))) or 
Vfc > i{s,k \= ip). 

We are particularly interested in temporal formulas that are valid with re- 
spect to the set of traces Sa associated with a GOAL agent A. Temporal 
formulas valid with respect to Sa express properties of the agent A. 

Definition 4.14 Let 5 be a set of traces. 

• 5'|=<^iffVsG5', i{s, i \= tp), 

• \= ip iff S \= ip where S is the set of all traces. 

In general, two important types of temporal properties can be distinguished. 

Temporal properties are divided into liveness and safety properties. Liveness 
properties concern the progress that a program makes and express that a (good) 
state eventually will be reached. Safety properties, on the other hand, express 
that some (bad) state will never be entered. In the rest of this section, we discuss 
a number of specific liveness and safety properties of an agent A = {ILa, Sq, Fq). 

We show that each of the properties that we discuss are equivalent to a set 
of Hoare triples. The importance of this result is that it shows that temporal 
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properties of agents can be proven by inspection of the program text only. The 
fact that proofs of agent properties can be constructed by inspection of the 
program text means that there is no need to reason about individual traces of 
an agent or its operational behaviour. In general, reasoning about the program 
text is more economical since the number of traces associated with a program 
is exponential in the size of the program. 

The first property we discuss concerns a safety property, and is expressed 
by the temporal formula (p {(f until tp). Properties in this context always 
refer to agent properties and are evaluated with respect to the set of traces 
associated with that agent. Therefore, we can explain the informal meaning of 
the property as stating that if (p ever becomes true, then it remains true until 
ip becomes true. By definition, we write this property as (fi unless ip: 

(fi unless ijj = (fi ^ {(p until ip) 

An important special case of an unless property is tp unless false, which 
expresses that if ever becomes true, it will remain true. <f unless false means 
that (/? is a stable property of the agent. In case we also have init ip, where 
init denotes the initial starting point of execution, ip is always true and is an 
invariant of the program. 

Now we show that unless properties of an agent A = (11, (To:7o) are 
equivalent to a set of Hoare triples for basic actions in 11. This shows that we 
can prove unless properties by proving a finite set of Hoare triples. The proof 
relies on the fact that if we can prove that after executing any action from 11 
either tp persists or tp becomes true, we can conclude that (p unless tp. 

Theorem 4.15 Let A = {IIa,T,o,Tq). Then: 

V 6 e I1a{{p> a -'tp} b {p>V Ip}) iff S'a 1= unless ip 

Proof: The proof from right to left is the easiest direction in the proof. Sup- 
pose Sa \= p unless tp and s,i \= p. This implies that s,i \= p until tp. In 

case we also have s,i \= tp, we are done. So, assume s, i ^ -itp and action b is 
selected in the trace at state s^. From the semantics of until we then know 
that (fV tp holds at state s^+i, and we immediately obtain {p) A ^tp} b {pV tp} 
since s and i were arbitrarily chosen trace and time point. To prove the Hoare 
triple for the other actions in the agent program A, note that when we replace 
action b with another action c from H^ in trace s, the new trace s' is still a 
valid trace that is in the set Sa- Because we have Sa \= p unless tjj, we also 
have s', i \= p unless tp and from reasoning by analogy we obtain the Hoare 
triple for action c (and similarly for all other actions). 

We prove the left to right case by contraposition. Suppose that 

(*) V 6 G IIa{{p> a ^tp} b {piV tp}) 

and for some s G Sa we have s,i ^ p unless tp. The latter fact means that we 
have s,i \= p and s,i ^ p until tp. s,i p until tp implies that either (i) tp 
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is never established at some j > i but we do have -up at some time point k > i 
or (ii) ip is estabhshed at some time j > i, but in between i and any such j it 
is not always the case that cp holds. 

In the first case (i), let A; > i be the smallest k such that s,k ^ (p. Then, 
we have s,k — 1 \= (p A -rip and s, fc ^ -^ip A ^tp. In state Sfc_i, however, either 
a conditional action is performed or no action is performed. From (*) we then 
derive a contradiction. 

In the second case (ii), \ei k > i be the smallest k such that s, A; |= ■(/;. Then 
we know that there is a smallest j such that i < j < k and s, j ^ (p {j ^ i since 
s, j 1= p). This means that we have s,j — 1 |= A However, in state Sj 
either a conditional action is performed or no action is performed. From (*) we 
then again derive a contradiction. 

Liveness properties involve eventualities which state that some state will be 
reached starting from a particular situation. To express a special class of such 
properties, we introduce the operator ip ensures ■(/>. tp ensures i\) informally 
means that guarantees the realisation of and is defined as: 

(/9 ensures il> = ip) unless A (i^ — > o-0) 

(p> ensures ?/; thus ensures that ■0 is eventually realised starting in a situation 
in which Lp holds, and requires that (p holds until is realised. For the class 
of ensures properties, we can show that these properties can be proven by 
proving a set of Hoare triples. The proof of a ensures property thus can be 
reduced to the proof of a set of Hoare triples. 

Theorem 4.16 Let A = (H^,cro,7o)- Then: 

V 6 e Ha({v? A ^V} & V tA}) A 3 & e i^aUp a h {il)}) 
^ Sa\= 'P ensures -0 



Proof: In the proof, we need the weak fairness assumption. Since Lp ensures -0 



is defined as p> unless ip A (ip ^ <>■!/;), by theorem 4.15 we only need to 
prove that Sa ^ ^ ^ oip given that V6 S TIa{{p A -^ip} b {pW ip}) A 3 6 G 
Hyi({(p A ^0} b {ip}). Now suppose, to arrive at a contradiction, that for some 
time point i and trace s G Sa we have: s,i \= p> A -iip and assume that for all 
later points j > i we have s,j \= -tip. In that case, we know that for all j > i 
we have s,j \= p A -iip (because we may assume p unless ip). However, we 
also know that there is an action b that is enabled in a state in which p A -iip 
holds and transforms this state to a state in which -0 holds. The action b thus is 
always enabled, but apparently never taken. This is forbidden by weak fairness, 
and we arrive at a contradiction. 



Finally, we introduce a third temporal operator 'leads to' The operator 
p ip differs from ensures in that it docs not require p to remain true until 
tp is established, and is derived from the ensures operator, is defined as 
the transitive, disjunctive closure of ensures . 
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Definition 4.17 (leads to operator) 
The leads to operator i-^- is defined by: 



(fi ensures ip 




(fl ^ i). ■ ■ ■ ,ipn ^ 




(f if) 


If tj) 


((ySl V . . . V (fn) - 





The meaning of the 'leads to' operator is captured by the following lemma. 
(f ip means that given ip condition tp will eventually be realised. The proof 
of the lemma is an easy induction on the definition of i— >. 

Lemma 4.18 y> i— > ■)/; |= y — > oip. 

5 Proving Agents Correct 

In this section, we use the programming logic to prove the correctness of our 

example shopping agent. We do not present all the details, but provide enough 
details to illustrate the use of the programming logic. Before we discuss what 
it means that an agent program is correct and provide a proof which shows 
that our example agent is correct, we introduce some notation. The notation 
involves a number of abbreviations concerning names and propositions in the 
language of our example agent: 

• Instead of current-website{sitename) we just write sitename; e.g., we 
write Am.com and ContentCart instead of current-website{Am.com) and 

current_website{ContentCart) , respectively. 

• As before, the book titles The Intentional Stance and Intentions, Plans 
and Practical Reason that the agent intends to buy are abbreviated to T 
and / respectively. These conventions can result in formulas like B(T), 
which means that the agent is at the web page concerning the book The 
Intentional Stance. 

A simple and intuitive correctness property, which is natural in this context 

and is applicable to our example agent, states that a GOAL agent is correct 
when the agent program realises the initial goals of the agent. For this sub- 
class of correctness properties, we may consider the agent to be finished upon 
establishing the initial goals and in that case the agent could be terminated. Of 
course, it is also possible to continue the execution of such agents. This class 
of correctness properties can be expressed by means of temporal formulas like 
Gcf) o-iG(t>. Other correctness properties are conceivable, of course, but not 
all of them can be expressed easily in the temporal proof logic for GOAL. 

5.1 Correctness Property of the Shopping Agent 

Prom the discussion above, we conclude that the interesting property to prove 
for our example program is the following property: 

Bcond A G{bought{T) A bought{I)) B{bought{T) A bought{I)) 
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where Bcond is some condition of the initial behefs of the agent. More specifi- 
cally, Bcond is defined by: 

Bcurrent-webpage{hpage{user)) A ^Bin_cart(T) A -iBm_cart(/)A 
B(Vs,s'((s ^ s' A current_webpage{s)) — > -icurrent_wehpage{s'))) 

The correctness property states that the goal to buy the books The Intentional 
Stance and Intentions, Plans and Practical Reason, given some initial conditions 
on the beliefs of the agent, leads to buying (or believing to have bought) these 
books. Note that this property expresses a total correctness property. It states 
both that the program behaves as desired and that it will eventually reach 
the desired goal state. An extra reason for considering this property to express 
correctness of our example agent is that the goals involved once they are achieved 
remain true forever (they are 'stable' properties). 



5.2 Invariants and Frame Axioms 

To be able to prove correctness, we need a number of frame axioms. There 
is a close relation between frame axioms and invariants of a program. This 
is because frame axioms express properties that are not changed by actions, 
and a property that, once true, remains true whatever action is performed is 
a stable property. In case such a property also holds initially, the property is 
an invariant of the program. In our example program, there is one invariant 
that states that it is impossible to be at two web pages at the same time: 
inv = BVs,s'((s ^ s' A current_webpage{s)) — > -^current_wehpage{s')). 

To prove that inv is an invariant of the agent, we need frame axioms stating 
that when inv holds before the execution of an action it still holds after executing 
that action. Formally, for each a G Cap, we need: {inv} a {inv}. These frame 
axioms need to be specified by the user, and for our example agent we assume 
that they are indeed true. By means of the Consequence Rule (strengthen the 
precondition of the Hoare triples for capabilities a) and the Rule for Conditional 
Actions (instantiate Lp and with inv), we then obtain that {inv{ b {inv} for 



all & G n. By theorem 4.15, we then know that inv unless false. Because we 
also have that initially inv holds since ((Jo, 70) |= inv, we may conclude that 
init Binv A inv unless false, inv thus is an invariant and holds at all times 
during the execution of the agent. Because of this fact, we do not mention inv 
explicitly anymore in the proofs below, but will freely use the property when 
we need it. 

A second property that is stable is the property status (book): 

status{book) = {Bin_cart{book) A Qbought[book)) V Bbought[book) 

The fact that status {book) is stable means that once a book is in the cart and 
it is a goal to buy the book, it remains in the cart and is only removed from the 
cart when it is bought. 
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The proof obligations to prove that status(hook) is a stable property, i.e. to 
prove that status{book) unless false, consist of supplying proofs for 



{status (book)} b {status (book)} 



for each conditional action & £ 11 of the shopping agent (cf. theorem 4.15). By 
the Rule for Conditional Actions, therefore, it is sufficient to prove for each 
conditional action — > c?o(a) G 11 that { status (book) A ip} a {status (book)} 
and {status{book) A -iTp) — > status{book). The latter implication is trivial. 
Moreover, it is clear that to prove the Hoare triples it is sufficient to prove 
{status{book)} a {status{book)} since we can strengthen the precondition by 
means of the Consequence Rule. The proof obligations thus reduce to prov- 
ing {status{book)} a {status{book)} for each capability of the shopping agent. 

Again, we cannot prove these Hoare triples without a number of frame ax- 
ioms. Because no capability is allowed to reverse the fact that a book has been 
bought, for each capability, we can specify a frame axiom for the predicate 
bought: 

(1) {^bought{book){ a {Bbought{book)} 

In case the book is not yet bought, selecting action pay_cart may change the con- 
tents of the cart and therefore we first treat the other three actions goto-website, 
search, and put_in_shopping_cart which are not supposed to change the contents 
of the cart. For each of the latter three capabilities we therefore add the frame 
axioms: 

{Bin_cart{book) A -^B bought (book)} a {Bin_cart{book) A -^B bought (book)} 

where a ^ pay_cart. Note that these frame axioms do not refer to goals but only 
refer to the beliefs of the agent, in agreement with our claim that only Hoare 
triples for belief updates need to be specified by the user. By using the axiom 
Gbought{book) -^Bbought{book) and the Consequence Rule, however, we can 
conclude that: 

{Bin_cart(book) A Gbought{book)} a {Bin_cart(book) A ^B bought (book)} 

By combining this with the axiom 

{Gbought{book)} a {Bbought{book) V Gbought{book)} 

by means of the Conjunction Rule and by rewriting the postcondition with the 
Consequence Rule, we then obtain 

{2){Bin-cart{book) A Gbought{book)} a {Bin_cart{book) A Gbought{book)} 

where a ^ pay-cart. By weakening the postconditions of (1) and (2) by means of 
the Consequence Rule and combining the result with the Disjunction Rule, it is 
then possible to conclude that {status {book)} a {status {book)} for a =^ pay-cart. 
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As before, in the case of capability pay_cart we deal with each of the disjuncts 
of status{hook) in turn. The second disjunct can be handled as before, but the 
first disjunct is more involved this time because pay_cart can change both the 
content of the cart and the goal to buy a book if it is enabled. Note that pay_cart 
only is enabled in case BContentCart holds. In case BContentCart holds and 
pay-cart is enabled, from the effect axiom for pay-cart and the Consequence 
Rule we obtain 

{Bin-cart{book) A Gbought{book) A BContentCart} 

(3) pay_cart 
{B bought (book)} 

In case -^BContentCart holds and pay-cart is not enabled, we use the Rule for 
Infeasible Capabilities to conclude that 

{Bin_cart{book) A Qbought{book) A -^BContentCart} 

(4) pay_cart 

{Bin_cart(book) A Gbought{book) A -^BContentCart} 

By means of the Consequence Rule and the Disjunction Rule, we then can 
conclude from (1), (3) and (4) that {status (book)} pay_cart {status (book)}, and 
we are done. 

5.3 Proof Outline 

The main proof steps to prove our agent example correct are listed next. The 
proof steps below consists of a number of ensures formulas which together 
prove that the program reaches its goal in a finite number of steps. 

(1) Bhpage{user) A -^Bin_cart{T) A Gbought{T)A 
A-'Bin-cart{I) A Gbought{I) 

ensures 

BAm.com A -^Bin-cart(T) A Gbought{T) A -^Bin-cart[I) A Gbought{I) 

(2) BAm.com A -'Bin^cart^T) A Gbought{T) A -'Bin-cart{I) A Gbought{I) 
ensures 

[(B(r) A Gbought{T) A -^Bm_cart{I) A Gbought{I))V 
(B(/) A Gbought{I) A -^Bin-cart{T) A Gbought{T))\ 

(3) B(T) A Gbought{T) A -^Bin^cart^I) A Gbought{I) 
ensures 

Bin-caTt{T) A Gbought{T) A -^Bin-cart{I) A Gbought{I) A BContentCart 

(4) Bin-cart{T) A Gbought{T) A -^Bin-cart{I) A Gbought{I) 
ensures 

BAm.com A ^Bin-cart{I) A Gbought{I) A status{T) 
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(5) B{Am.com) A -■Bm_cart(/) A Gbought{I) A status{T) 
ensures 

B(/) A Qhought{I) A status{T) 

(6) B(/) A Qhought{I) A status {T) 
ensures 

Bin-cart{I) A Gbought{I) A BContentCart A status{T) 

(7) B?;n_cart(/) A Gbought{I) A BContentCaH A status (T) 
ensures 

Bbought{T) A Bbought{I) 

At step 3, the proof is split up into two subproofs, one for each of the dis- 

juncts of the disjunct that is ensured in step 2. The proof for the other disjunct 
is completely analogous. By applying the rules for the 'leads to' operator the 
third to seventh step result in: 

(a) B{T) A Qbought{T) A -.Bm_cart(/) A Gbought{I) ^ 

Bbought{T) A Bbought{I) 
(6) B{I) A Gbought{I) A -.Bm_cari(T) A Gbought{T) i-^ 

Bhought{T) A Bhought{I) 

Combining (a) and (b) by the disjunction rule for the 'leads to' operator and by 
using the transitivity of 'leads to' we then obtain the desired correctness result: 

Bcond A G{bought{T) A hought{I)) ^ B{bought{T) A bought{I)) 

with Bcond as defined previously. 

Step 1 We now discuss the first proof step in somewhat more detail. The re- 
mainder of the proof is left to the reader. The proof of a formula ip ensures tp 
requires that we show that every action b in the Personal Assistant program 
satisfies the Hoare triple {tp A -^tp} b {(fV ijj} and that there is at least one ac- 
tion b' which satisfies the Hoare triple {(p A -^t/j} b' {V^}. By inspection of the 
program, in our case the proof obligations turn out to be: 

{Bhpage{user) A -^Bin-cart{T) A Gbought{T) A -•Bm_cart(/) A Gbought{I)} 
b 

{Bhpage{user) A -'Bin-cart{T) A Gbought{T) A -^Bm-cart{I) A Gbought{I)} 
where b is one of the actions 

B{Ain.com) A -'B{in_cart{book)) A G{bought{book)) do{search{book)), 
B{book) A G{bought{book)) — > do{put-in-shopping-cart{book)), 
B{in-cart{book)) A G{bought{book)) — > do{pay-cart)} 

and 
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{Bhpage(user) A -^Bin_cart{T) A Gbought(T) A -^Bin_cart{I) A Gbought{I)} 

B{hpage{user) V ContentCart) A G{bought{book)) — > do{goto-website{Am.com)) 
{BAm.com A -^Bin_cart{T) A Q>bought{T) A -^Bin_cart{I) A Gbought(I)} 

The proofs of the first three Hoare triples are derived by using the Rule for 
Conditional Actions. The key point is noticing that each of the conditions of 
the conditional actions involved refers to a web page different from the web page 
hpage{user) referred to in the precondition of the Hoare triple. The proof thus 
consists of using the fact that initially Bhpage{user) and the invariant inv to 
derive an inconsistency which immediately yield the desired Hoare triples by 
means of the Rule for Conditional Actions. 

To prove the Hoare triple for 
B{hpage{user)\/ ContentCart) A G{bought (book)) — > do{goto-website{Am.com)) 
we use the effect axiom (5) for goto_website and the frame axiom (6): 

{Bhpage{user)} 

(5) goto-website(Am.com) 
{BAm.com} 

and 

{-^Bin_cart{book) A -'Bbought(book)} 

(6) goto_website{Am.com) 
{-^Bin_cart{book) A -'Bbought(book)} 

By using the axiom 

{Gbought{book)} goto_website{Am.com) {Bbought{book) V Gbought{book)} 

the Conjunction Rule and the Rule for Conditional Actions it is then not difhcult 
to obtain the desired conclusion. 

6 Possible Extensions of GOAL 

Although the basic features of the language GOAL are quite simple, the pro- 
gramming language GOAL is already quite powerful and can be used to program 
real agents. In particular, GOAL only allows the use of basic actions. There are, 
however, several strategies to deal with this restriction. First of all, if a GOAL 
agent is proven correct, any scheduling of the basic actions that is weakly fair 
can be used to execute the agent. More specifically, an interesting possibility is 
to define a mapping from GOAL agents to a particular agent architecture (cf. 
also Q). As long as the agent architecture implements a weakly fair schedul- 
ing policy, concerns like the efficiency or flexibility may determine the specific 
mapping that is most useful with respect to available architectures. 

A second strategy concerns the grain of atomicity that is required. If a 
coarse-grained atomicity of basic actions is feasible for an application, one might 
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consider taking complex plans as atomic actions and instantiate the basic actions 
in GOAL with these plans (however, termination of these complex plans should 
be guaranteed). Finally, in future research the extension of GOAL with a richer 
notion of action structure like for example plans could be explored. This would 
make the programming language more practical. The addition of such a richer 
notion, however, is not straightforward. At a minimum, more bookkeeping 
seems to be required to keep track of the goals that an agent already has chosen 
a plan for and which it is currently executing. This bookkeeping is needed, for 
example, to prevent the selection of more than one plan to achieve the same 
goal. Note that this problem was dealt with in GOAL by the immediate and 
complete execution of a selected action. It is therefore not yet clear how to give 
a semantics to a variant of GOAL extended with complex plans. 

The ideal, however, would be to combine the language GOAL which includes 
declarative goals with our previous work on the agent programming language 
3APL which includes planning features into a single new programming frame- 
work. Let us elaborate a little on the (im-)possibilities, here. A 3APL-Goal is a 
program or procedural goal, written P-Goal, and defined as either a basic action 
Bact C Goal, a test on the beliefs of the agent (^? or composed as a sequence 
(""i; 7r2) or a choice (tti + ^2). However, also goal- variables X are allowed in 
P-Goals. Central in 3APL is the so-called practical reasoning rule of the form 

TTh ^ f I VTfc e 

which should be read as: 'if tt^, the goal in the head of the rule, is the agent's 
current (procedural) P-Goal, and he believes that ip is the case, then the rule 
allows the agent replace nh with the goal in the body, TTf,' 

Apart from introducing more complex action structures, it would also be par- 
ticularly interesting to extend GOAL with high-level communication primitives. 
Because both declarative knowledge as well as declarative goals arc present in 
GOAL, communication primitives could be defined in the spirit of speech act 
theory |29t] . The semantics of, for example, a request primitive could then be 
formally defined in terms of the knowledge and goals of an agent. Moreover, 
such a semantics would have a computational interpretation because both beliefs 
and goals have a computational interpretation in our framework. 

Finally, there are a number of interesting extensions and problems to be 
investigated in relation to the programming logic. For example, it would be 
interesting to develop a semantics for the programming logic for GOAL that 
would allow the nesting of the belief and goal operators. In the programming 
logic, we cannot yet nest knowledge modalities which would allow an agent 
to reason about its own knowledge or that of other agents. Moreover, it is 
not yet possible to combine the belief and goal modalities. It is therefore not 
possible for an agent to have a goal to obtain knowledge, nor can an agent have 
explicit rather than implicit knowledge about its own goals or those of other 
agents. So far, the use of the B and G operators in GOAL is, first of all, to 
distinguish between beliefs and goals. Secondly, it enables an agent to express 
that it does not have a particular belief or goal (consider the difference between 
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-iBcf) and B-i(/)). Another important research issue concerns an extension of the 
programming framework to incorporate first order languages and extend the 
programming logic with quantifiers. Finally, more work needs to be done to 
investigate and classify useful correctness properties of agents. In conclusion, 
whereas the main aim may be a unified programming framework which includes 
both declarative goals and planning features, there is still a lot of work to be 
done to explore and manage the complexities of the language GOAL itself. 

7 Conclusion 

Although a programming language dedicated to agent programming is not the 
only viable approach to building agents, we believe it is one of the more practical 
approaches for developing agents. Several other approaches to the design and 
implementation of agents have been proposed. One such approach promotes the 
use of agent logics for the specification of agent systems and aims at a further 
refinement of such specifications by means of an associated design methodology 
for the particular logic in use to implementations which meet this specification 
in, for example, an object-oriented programming language like Java. In this 
approach, there is no requirement on the existence of a natural mapping relating 
the end result of this development process - a Java implementation - and the 
formal specification in the logic. It is, however, not very clear how to implement 
these ideas for agent logics incorporating both informational and motivational 
attitudes and some researchers seem to have concluded from this that the notion 
of a motivational attitude (like a goal) is less useful than hoped for. 

Still another approach consists in the construction of agent architectures 
which 'implement' the different mental concepts. Such an architecture pro- 
vides a template which can be instantiated with the relevant beliefs, goals, etc. 
Although this second approach is more practical than the first one, our main 
problem with this approach is that the architectures proposed so far tend to 
be quite complex. As a consequence, it is quite difficult to understand what 
behaviour an architecture that is instantiated will generate. 

For these reasons, our own research concerning intelligent agents has focused 
on the programming language 3APL which supports the construction of intelli- 
gent agents, and reflects in a natural way the intentional concepts used to design 
agents (in contrast with the approach discussed above which promotes the use 
of logic, but at the same time suggests that such an intermediate level is not 
required). 

Nevertheless, in previous work the incorporation of declarative goals in agent 
programming frameworks has, to our knowledge, not been established. It has 
been our aim in this paper to show that it is feasible to incorporate declarative 
goals into a programming framework (and there is no need to dismiss the con- 
cept). Moreover, our semantics is a computational semantics and it is rather 
straightforward to implement the language, although this may require some 
restrictions on the logical reasoning involved on the part of GOAL agents. 

Let us briefly indicate how incorporating declarative goals in the language 
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3APL might proceed. To this end, let us rename the goals in 3APL to plans tt, 
which are either a basic action a(ti, ...,<„) on terms U, a test ip7 or combined in 
sequential composition (tti; 112) or nondeterministic choice (tti + ^2). One may 
also use goal-variables — X, l", ... in goals. Central in 3APL are the so-called 
Practical Reasoning Rules, in its most general form written as 

TT/i <— "yS I TTfc G Rule 

In this paper, we provided a complete programming theory. The theory 
includes a concrete proposal for a programming language and a formal, opera- 
tional semantics for this language as well as a corresponding proof theory based 
on temporal logic. The logic enables reasoning about the dynamics of agents 
and about the beliefs and goals of the agent at any particular state during its 
execution. The semantics of the logic is provided by the GOAL program se- 
mantics which guarantees that properties proven in the logic are properties of 
a GOAL program. By providing such a formal relation between an agent pro- 
gramming language and an agent logic, we were able to bridge the gap between 
theory and practice. Moreover, a lot of work has already been done in providing 
practical verification tools for temporal proof theories |Q . 

Finally, our work shows that the (re)use of ideas and techniques from con- 
current programming can be very fruitful. In particular, we have used many 
ideas from concurrent programming and temporal logics for programs in devel- 
oping GOAL. It remains fruitful to explore and exploit ideas and techniques 
from these areas. 
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